MacPhail 


MacPhail Online School 
Partnerships 


2018-19 Evaluation Results 


Authors: Emma Connell and Julia Miller 


OCTOBER 2019 Wilder 
Research, 


451 Lexington Parkway North | Saint Paul, Minnesota 55104 
Information. Insight. Impact. 


651-280-2700 | www.wilderresearch.org 


Contents 


IPOS Pe VEC W aaa sdecavanteanscinndenae estendeuredeausneracsenteatoneateas coaaeeaedeae nae 1 
MEGA Eset nese tertewoe cae aantcotemealeges os tae ciotanteor eaten ee ateocos seu statee wines aes ecenereecareeeeeo eee > 
Implementation evaluation key QUestiONS.............ccccesccesceeeeesseceseceeeeeeeeeeaeecsaeceeeeeeeeenees ) 
CIOIC OME CV Al AION REY QUES ONS irs2iadusesavcsusceduuremnaabonisadnudundsnetnededurbeonavanindanevaoniees 3 
TSU S 2 fotice-icente dis menae seas cacsvemcascesuceuncstatncatoneisnsusine.ceseate aces etre niesudaraiceem nearer ce S 
SSSI ces seuaii sia atoacusa ola area inte mses saan mam acemcan ula ees ease eons ana exd i iaasuneaeoeoaeaanicaeeataSaae a 
Mp1 AU OIA occas tues seinen beripatpwennensbsisons sea pacisu seen atyiisesaisanensdadtu sendin cabanopsnhecsseanweailices 5 
CNUUCOTINGS: 5 sas 22es tas necetisetaaensa ven tatatitalcasutceesacevsesdssueced ate sstedesdiuecucteasiauentenat attalasbnuaiea 7 
SSUES 10 -COMSIGEE sao siacvcts Gace sanctateasusianaaealaesneasstainane mb iexesttceeaoa nage 10 
2 0) | ch eee re ee Re ene me terete ere er ene eee eee ae any eR one ae Cerne eT een eee ‘ial 
Temp lemenita Hi OM zsks ac cinccsaneso vvessebiseedsanrdcsantnaccisteas tern darece Camesmmnanons erie ncameates 11 
COE ope setae tip nest deadcpese cesses hibtedseud ie teaslxeadenanuen seed iaediaud reeset ead nade auxae ieiccsausareduce 12 
AU) 0 [cli (0b, eee en eae ee eee ae et eee ere oe ean er Seen eee rer eee eee, 16 
pened COMTI ¢ <cckvucasasnahcateirauceonrniacteasinuaiun eo eadesanenasaals 16 
LOB 1G MOTE sazprsshscveansuracsustvesesrsunndsursesesdvenesexenatipedesussandustaniustneconnsyscecsesbonisnbuntertuauaneces 18 
MacPhail Online School Partnerships Logic Model..........c.cccesscceseceeeeeeeeeeseeeeeceteeeees 18 
SVS TS ERIS TTS ice ic coda vaaainsaceas Sone ealeeceenrsa noone een es 19 


MacPhail Online School Partnerships 2018-19 Evaluation Results Wilder Research, October 2019 


Figures 


Te DMS teal let er yeh ce cacevosndecceuateassiwendacasncaacecoteatuu tenteanadomneeaaneateneetnadawn 8 
Do MCAT OOS scoters cinaid Rer ee teceeebn dene eee ac ecsas dates teres eevdace ceed emi dctaseepeacg leds 8 
De AMG Stal Atte s.2c.icasscascassiarcaavensducnapiadacsesbacteseplancemianearia i spenaapncnasiaiieannaunmianenials 9 
4. Students’ general satisfaction with the Online School Partnerships program.......... 11 
5. Students’ satisfaction with technology Used ...........ecceccceeseeesceceeceeeeeeeeesseecsaeeneenees 11 
Ge. SUVGENts TEPOE, OF NSIT UCTOL .ciscciaccseeitaradanissxianiniataeniaensseniienateiiaexperniattailastandeel: 11 
7. Student satisfaction with Online School Partnerships program classes...............0++ 12 
8. Student suggestions for program iIMProvemMent ............c ce ceceeceeceeeeesseceteceeeeeeeeeeaees 12 
9. Students’ report of musical skills and knowledge .........cceceecceceseceeeeeeeeeeeeeseeeteenees 12 
10. Tone quality before and after online SeSSIONS............ceececeseceteceeteceteeeeeeeeseeceseeneeeees 12 
11. Intonation before and after online SeSSIONS............ ce eeeeseeeeceeeceeecneeeseeeeeeeeeeaeeeaeenteess 13 
12, Rhythm before and atter Online SESSiOMS csssicssssecasssdsiecscsoedavesabiieontevnszcovesddicensseioveds 13 
13. Technique before and after online SESSIONS .0.........:eecceecceeseeesceceeceeeeeeeeeeseecsaeeneeeees 13 
14. Musical expression before and after online SeSSIONS ...........ceceeeceeeeeeeetseeeeeeteeeees 13 
1D), SOWIE (CS HANCS) scxcuasevssesecnsedentiabastswusaacaessessapauenersehaabestsecoensssissescetesgentswubestweuaetes 13 
DiS. FER CCUTOM a ceiaviess; sicciesidacasiabacccaniug acs sdecsasheeseas haieuaaindemaagiatiaatsniaeeaa nade eaaomaee 13 
IF“, PASTAS UA cede seem tse aceins busch a acai nea ene Oo tee ee 14 
Wye SAIS SCI a cca ces ecscseesnaenserasiesacesuncasstcuatestereacsaneenapeatscamsneerourncenuntaeruseocatne: 14 
VD, COMPA ICS 5 sicsus siccasncenenendpadenonnvaccnspasaceaatianceaa piacere ieanteaasinincaentaema nadecmeaaaeniageaas 14 
AU) - PROUT TO: FOCUS: cisce sussususanelundslivvcusseasaatuocatosubeacsasempswesubbuncaitencasisenercsosnistvuunshaaiie@onaes 14 
21... AS OMAP SLE PAT CHC HALTOM osisicsisics cuceascsnaesievsvicnsean ern casaces und esewenncawestenlacganmrencasuuseateneariacs 14 
22, Types of musical competitions participated ih sosccnscssacvsssnedecnstensanasteadeeessanienadondenaes 14 
23... PUSS TSDOM, Ol MUSICA! TMSLESE cies cxsteecssvudsnnssenvsnedushowndcvedesdudssosmnsaicnrseadutucntaves 15 
24... Students’ report of work 1: SCHOO] dss, sscsescsacsignageisedisneeoadenaniaioanbiancariaiae 15 


MacPhail Online School Partnerships 2018-19 Evaluation Results Wilder Research, October 2019 


Program overview 


The MacPhail Center for Music Online School Partnerships program aims to increase 
access to high-quality music education for students who attend schools in Greater 
Minnesota in order to improve student interest and knowledge in music, and to improve or 
maintain positive school engagement and non-academic skills. The MacPhail Teaching Artists 
work collaboratively with school music educators to develop curriculum, implement the 
Online School Partnerships program, and work with students individually, in sectionals, or 
in large group settings on their music lessons via the internet. Teaching Artists teach classes 
at the partner schools remotely and in real-time through teleconferencing technology. Classes 
are typically 30-60 minutes in length, with a MacPhail Teaching Artist working with students 
on their repertoire for their ensemble or for an audition, as well as helping students as they 
develop their musical skills. All of the session content is based on what the partner schools’ 
music educator would like the Teaching Artist to cover — the Teaching Artist then creates 
the session structure. 


The Online School Partnerships program has been providing lessons and working with 
schools outside the Twin Cities metro region since 2011. In 2018-19, it had 30 Teaching 
Artists, partnered with 21 schools, and served more than 800 students. Schools involved in 
the partnership had a wide range of instruction, between eight and 348 hours each, totaling 
1,479 hours of instruction. This evaluation, however, only focuses on 10 schools who 
chose to participate in the evaluation in the 2018-19 school year: Braham Area High 
School, Buffalo High School, Detroit Lakes High School, Kerkhoven-Murdock-Sunburg 
High School, Kingsland Senior High School, Tri-County Public Schools, MACCRAY 
High School, Vermilion Country School, Willmar High School, and Yellow Medicine 
East High School. Twenty-two Teaching Artists from MacPhail were involved in the 
Online School Partnerships program for the schools participating in this evaluation. 
This evaluation focuses on individual and small group lessons with a MacPhail Teaching 
Artist. The types of lessons provided by the Online School Partnerships program are 
described as follows: 


= Group lessons: These lessons are ongoing and built on method books, handouts, and 
sequential practice planning. The MacPhail Teaching Artists consult with the partnering 
music educators to create ongoing assignments that encourage musicianship and solid 
playing techniques for the group of students, with a focus on one particular instrument 
or voice type. 
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= Sectionals: In these lessons, MacPhail Teaching Artists work to boost the skills of 
specific sections of choir, band, or jazz bands. They work on repertoire the participating 
students are playing in large group. These students work on development of sound, 
rhythmic precision, general musical understanding, and performance skills. 


= Individual lessons: These lessons offer one-on-one instruction with an instrument or 
vocal specialist to work on general tone and technique. Teaching Artists may also 
provide support as a student prepares for All-State band or choir and contest 
preparation. 


Seventy students took lessons through the Online School Partnerships program at the 
schools involved in this evaluation. These schools had between twelve and 347 hours of 
instruction, totaling 1,213 hours of instruction throughout the 2018-19 school year. The 
average number of instruction hours at these schools was 121.35 hours, and the median was 
65.75 hours. All of the students involved in this evaluation took individual lessons from 
Teaching Artists through the Online School Partnerships program. 
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Evaluation 


Wilder Research worked with the Online School Partnerships program to develop an 
evaluation design for the program. The evaluation includes implementation and outcome 
components, and gathers qualitative and quantitative data to answer the following research 
questions. (The Online School Partnerships program developed a logic model and these 
research questions reflect program activities and outcomes. See Appendix). 


Implementation evaluation key questions 


= How well is the Online School Partnerships program being implemented? What are 
aspects of the program, if any, that need to be adjusted over time? 


= How well does the collaboration between MacPhail and the schools function, and 
how can it be strengthened? 


Outcome evaluation key questions 


= To what extent does student participation in the Online School Partnerships program 
affect their interest, motivation, knowledge, and skills in music? 


= To what extent does student participation in the Online School Partnerships program 
affect their attitudes about school (school motivation and engagement) and student 
non-academic skills? 


= To what extent does music educator participation in the Online School Partnerships 
program affect their growth as an educator, their job satisfaction, and ability to handle 
many aspects of their work assignment? 


Methods 


MacPhail and Wilder Research staff worked collaboratively to complete this evaluation. Both 
MacPhail and Wilder staff completed data collection responsibilities, and Wilder Research 
staff completed data entry, data analysis, and reporting. 


The following are the data collection instruments used in the evaluation. 


m= Teaching Artist survey: In spring 2019, Wilder conducted a web survey with the 
MacPhail Teaching Artists. This survey asked about the overall vision and goals of 
the program, the collaboration with the partner schools, successes of the Online 
School Partnerships program, and areas for improvement. Twenty Teaching Artists 
completed the survey, for a response rate of 91%. 


MacPhail Online School Partnerships 2018-19 Evaluation Results 3 | Wilder Research, October 2019 


m= Student survey: This survey was developed to measure student satisfaction with the 
Online School Partnerships program, perceptions of the lessons provided, and interest 
in music education and school generally. In spring 2019, nine seventh- through 
twelfth-graders completed a survey, representing 13% of the 70 students in the program 
at these schools. 


= Music skills assessments: At the beginning and end of students’ time with the Online 
School Partnerships program, the MacPhail Teaching Artist they worked with 
completed skills assessments, looking at a variety of components of students’ 
performance abilities (tone quality, intonation, rhythm, etc.). Pre-post music skill 
assessments were completed for 64 of 70 students (91%) involved in the Online 
School Partnerships program from Braham Area High School, Buffalo High School, 
Detroit Lakes High School, Kingsland Senior High School, Tri-County Public Schools, 
MACCRAY High School, Vermilion Country School, Willmar High School, and 
Yellow Medicine East High School. 
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Results 


This section describes the implementation and outcome results. Results are reported across 
the schools, and separately when appropriate. Results should be interpreted with caution 
given the small sample size. 


Implementation 


Successes 


Students enjoy their lessons and working with Teaching Artists. 


Students were asked about their satisfaction with the program in a survey administered at 
the end of the school year. All students agreed that they liked coming to the online sessions, 
and eight of the nine responding students said they would recommend the sessions to 
other students (Figure 6). All students agreed that the technology used for the sessions 
made it easy to learn, though fewer students identified that they strongly agree than on 
many other questions (Figure 7). 


Similarly, all responding students reported that they liked the Teaching Artist who led the 
class and that their Teaching Artist spent the time well (Figure 8). Eight of the nine students 
reported that their Teaching Artist set clear expectations for practicing and learning at home. 


Students were asked in an open-ended question what they liked most about participating 
in the Online School Partnerships program. Most commonly, students spoke about the 
convenience or accessibility of the lessons and how they gained new skills and knowledge 
or improved their technique by participating in their sessions (Figure 9). Most students 
could not think of suggestions for the program when asked (Figure 10). Two students 
noted that the technology used for the sessions could be improved, though no specific 
suggestions were provided. 
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MacPhail is a resource and partner to schools in Greater Minnesota. 


Teaching Artists commonly spoke about how the Online School Partnerships program 
increases access for students who otherwise would not otherwise have access to 
specialized and high-quality instruction. Students in Greater Minnesota lack the same 
access to this type of musical instruction that students in the Twin Cities have, as they 
typically only have access to school-based music instructors, who must be generalists. This 
musical instruction opens up opportunities for other benefits, such as improved musical 
skills, new relationships, and a broader understanding of what students’ musical futures 
might hold. 


I believe the biggest success for the OSP program is the amount of arts access 
that it has been able to grant to the surrounding Minnesota communities and the 
children that live in those communities. — Teaching Artist 


This is an outstanding resource for band directors around the state. Sometimes 
it’s impossible for band directors to find an expert to an instrument to help his 
students. — Teaching Artist 


More broadly, some Teaching Artists also highlighted the importance of MacPhail being 
known as a resource and willing partner for schools in Greater Minnesota. 


I think the biggest successes are the relationships that have been established. 
MacPhail [is] becoming a known resource for schools outside the metro area. 
The individual accomplishments of the students who had benefited from the 
partnership are of course important, but establishing the longer-term 
relationships with those schools and, in turn, the schools they pass the word on to 
is, I feel, the biggest success so far. — Teaching Artist 


Opportunities for improvement 


High-quality and consistent technology continues to be a struggle for schools. 


Teaching Artists were asked how the program could improve. Similar to previous years, 
respondents most frequently discussed issues around the technology used for lessons. 
Technology problems result in lost lesson time and frustration for both the Teaching 
Artist and the student. 


Some technical problems. Transmission in buried practice rooms in remote 
locations is sometimes dicey — some lessons are spent re-establishing contact and 
then resorting sometimes to FaceTime sessions on students’ devices. 

— Teaching Artist 
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Improved communication can improve program impact. 


Some Teaching Artists discussed opportunities for better communication between 
MacPhail, Teaching Artists, and participating schools. Teaching Artists discussed 
challenges with getting full buy-in from students when the music educators at their 
school are not fully engaged in the program. 


When communication and involvement from the band directors is minimal, it is 
difficult to engage the students fully. — Teaching Artist 


Programmatic recommendations varied. 


Teaching Artists provided a wide range of ideas to improve the impact of the program. 
These included having incentives or goals for students, providing group programming, 
holding occasional face-to-face lessons, providing lessons during the summer, collaborating 
with the School Partnerships work of MacPhail, and making programming more 
consistent across schools. 


Consistency of programming is an area that could be improved upon. This would 
greatly benefit from longevity of the program in various schools... if the OSP 
program had solid funding from the state or otherwise, it would allow for the 
programs to consistently build in each district. — Teaching Artist 


More offerings in the summer. —Teaching Artist 


Outcomes 
Benefits to students in music 


Students gain musical skills and knowledge. 


When students were asked to how much they agree with the statement “I learned new 
musical skills that helped me in band/choir,” students were positive; eight of nine 
respondents reported that they “strongly agree” with the statement (Figure 11). 


Teaching Artists assessed students’ musical skills at the beginning and end of their time 
with the Online School Partnerships program. Students were assessed in a variety of areas: 
tone quality, intonation, rhythm, technique, musical expression, execution (for percussion 
students), articulation (for wind students), and music reading skills (Figures 12-20). 


Overall, students showed moderate growth in most skills. Students on average grew .5 
points in five areas: bowing, technique, rhythm, articulation, and reading skills (where 
1=poor, 2=fair, 3=good, 4=excellent, and 5=superior; Figure 1). These were the areas 
with the most improvement. 
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1. Musical skill growth 


Rhyth) iT 3.3 
Articulation (Ts 3.7 
Intonation TT 3.7 
Musical expression (ITs 37 
Reading skils (IT 3.7 
Execution (i 3.6 
Technique ST 3. 
Tone quality i 3.c 
Bowing 3. 


Note. This chart shows change in mean scores for all Online School Partnerships program participants included in this 
evaluation from pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


On average, Students were rated highest in rhythm (3.8), followed by articulation, reading 
skills, intonation, and musical expression (3.7 in all; Figure 2) in post assessments. 
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Note. This chart shows mean scores at the end of lessons for all Online School Partnerships program participants 
included in this evaluation from pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


MacPhail Online School Partnerships 2018-19 Evaluation Results 8 | Wilder Research, October 2019 


The music skills assessment also asked Teaching Artists to provide ratings in a student’s 
musical confidence and ability to focus (Figures 20 and 21). Students made improvements 
in mean scores of 0.5 points in confidence (from mean scores of 3.5 to 4.0) and 0.2 points 
in ability to focus (from mean scores of 4.1 to 4.3; Figure 3). 


3. Musical attitude 
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Note. This chart shows mean scores at the end of lessons for all Online School Partnerships program participants 
included in this evaluation from pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


Ability to focus 


Confidence 


Students were asked if they participated in any musical competitions or scholarship 
auditions in the previous school year. All nine respondents had participated in a musical 
competition, and four had participated in a scholarship competition (Figure 23). All nine 
responding students participated in Solo and Ensemble, six participated in Honor Band, 
and one student participated in All-State (Figure 24). 


Students’ musical skills, appreciation of music, and confidence in playing grew as a 
result of participation in the Online School Partnerships program. 


Students participating in the Online School Partnerships program have access to 
specialized and high-level musical instruction that otherwise would not be available to 
them. Teaching Artists described increased access to high-quality music instruction as 
one of the major successes of the program. This instruction leads to improved musical 
skills and increased confidence among participants. 


Helping students to advance their musicianship and give them the confidence to 
participate in musical activities that they might not otherwise have tried for, such 
as solo and ensemble contest, a collegiate music scholarship, or an All-State 
ensemble seem like worthwhile successes to me. — Teaching Artist 


Students show interest and motivation in music. 


All responding students reported they either “agree” or “strongly agree” that coming to the 
online sessions increased their appreciation of music (Figure 25). All students strongly 
agreed that they are interested in continuing to develop their musical skills. All students 
reported interest in participating in a college band or orchestra, seven of the nine responded 
that they are interested in pursuing a college degree in music, and eight of the nine 
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reported that they were interested in continuing their instrument on an informal basis 
after graduating high school. 


Students show increased interest in school. 


All students agreed to the statement “I work hard in school,” with eight of the nine strongly 
agreeing with the statement (Figure 26). Seven of the nine students reported that participating 
in the Online School Partnerships made them more motivated to do well in school. 


Issues to consider 


This evaluation of the Online School Partnerships program indicates that participants are 
doing well in the program. Students generally enjoy their classes and are actively engaged 
and progressing in their music education. Teaching Artists are satisfied with the program 
and see it as an asset for schools across Greater Minnesota. 


Based on the results available to date, the following are a few points to consider in 
future programming. 


Going forward, MacPhail could consider the following improvements: 


= Work with evaluators to find ways to increase response rates for the student survey. 


m= Work with evaluators to include more feedback from school principals, music 
educators, classroom teachers, and parents of program participants. 


= Work with evaluators to dig deeper into the technology aspect of programming; 
students replied that the technology used for programming made it easier to learn, 
though technology issues are the most common issues discussed for the program. 
Future evaluation could learn more about the benefits of the existing use of technology, 
difficulties in using it, and potential options for MacPhail and partner schools to 
explore. We understand that MacPhail is exploring a variety of alternative platforms, 
and will be testing a new platform in the coming year. 


m Encourage greater communication between MacPhail, Teaching Artists, and schools 
participating in the program 


m Expand access to the program by securing additional funding. We understand that the 
program has recently been granted funds from the Minnesota Department of Education; 
we anticipate this will assist greatly in expanding access. 


MacPhail Online School Partnerships 2018-19 Evaluation Results 10 | Wilder Research, October 2019 


Tables 


Implementation 
4. Students’ general satisfaction with the Online School Partnerships 
program 
Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
| like coming to the online sessions. 9 4/9 5/9 0/9 0/9 0/9 
| would recommend these sessions 9 6/9 2/9 0/9 0/9 1/9 
to other students. 
5. Students’ satisfaction with technology used 
Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
The technology used for these 3/9 6/9 0/9 0/9 0/9 
sessions made it easy to learn. 
6. Students’ report of instructor 
Strongly Strongly Don’t 
N agree Agree Disagree disagree know 
| liked the Teaching Artist who led the 8 6/8 2/8 0/8 0/8 0/8 
class. 
The Teaching Artist used class time 9 7/9 2/9 0/9 0/9 0/9 
well. 
The Teaching Artist set clear 9 6/9 2/9 0/9 0/9 1/9 


expectations for practicing and 
learning at home. 


Note. Totals may not equal 100 due to rounding 
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7. Student satisfaction with Online School Partnerships program classes 


What did you like most about this 


class? N=9 N Percent 
Convenience/accessibility of lessons 4 4/9 
New skills/technique/knowledge 4 4/9 
Experiences with Teaching Artist 2 2/9 
Other 1 1/9 


Notes. Students may have noted more than one thing they liked most about the class, resulting in a total >100%. 
Percentages given reflect the number of times a response was given divided by the total number of responses. 


8. Student suggestions for program improvement 


What would you improve? N=9 N Percent 
No suggestions 6 6/9 
Technology improvements 2 2/9 
Other 1 1/9 


Notes. Students may have noted more than one thing they liked most about the class, resulting in a total >100%. 
Percentages given reflect the number of times a response was given divided by the total number of responses. 


Outcomes 


9. Students’ report of musical skills and knowledge 


Strongly Strongly Don’t 
N agree Agree Disagree disagree Know 
| learned new musical skills that 9 8/9 1/9 0/9 0/9 0/9 
helped me in band/choir. 
10. Tone quality before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 64 13% 44% 41% 2% 2% 3.6 aa 
+0. 
Pre 64 14% 25% 47% 11% 3% 3.4 
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11. Intonation before and after online sessions 


Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 58 10% 50% 36% 2% 2% 3.7 oe 
+0. 
Pre 58 10% 35% 38% 16% 2% 3.4 
12. Rhythm before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 64 17% 48% 28% 5% 2% 3.8 nA 
+0. 
Pre 64 11% 31% 41% 14% 3% 3.3 
13. Technique before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 65 12% 42% 42% 3% 2% 3.6 ae 
+0. 
Pre 65 8% 28% 34% 29% 2% 3.1 
14. Musical expression before and after online sessions 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 64 20% 38% 36% 5% 2% 3.7 ae 
+0. 
Pre 64 17% 17% 30% 33% 3% 3.1 
15. Bowing (Strings) 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 8 0/8 2/8 5/8 1/8 0/8 3.1 ae 
+0. 
Pre 8 0/8 2/8 2/8 3/8 1/8 2.6 
16. Execution 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 64 16% 41% 38% 5% 2% 3.6 ba 
+0. 
Pre 64 8% 28% 42% 20% 2% 3.2 
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17. Articulation 


Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 64 16% 44% 38% 3% 0% 3.7 ae 
+0. 
Pre 64 8% 27% 41% 25% 0% 3.2 
18. Reading skills 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 59 29% 25% 37% 7% 2% 3.7 we 
+0. 
Pre 59 9% 31% 36% 22% 3% 3.2 
19. Confidence 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 65 26% 51% 19% 3% 2% 4.0 a 
+0. 
Pre 65 25% 26% 29% 17% 3% 3.5 
20. Ability to Focus 
Superior Excellent Good Fair Poor 
N 5 4 3 2 1 Mean Change 
Post 66 47% 38% 11% 5% 0% 4.3 ne 
+0. 
Pre 66 41% 35% 17% 8% 0% 4.1 
21. Competition participation 
Scholarship Musical 
N competitions competitions 
9 4/9 


22. Types of musical competitions participated in 


N Solo/Ensemble 


Honor Band 


All-State 


9 9/9 
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23. Students’ report of musical interest 


Strongly Strongly Don’t 

N agree Agree Disagree disagree know 
Coming to the online sessions 9 4/9 5/9 0/9 0/9 0/9 
increased my appreciation of music. 
| am interested in continuing to 9 9/9 0/9 0/9 0/9 0/9 
develop my musical skills. 
| am interested in pursuing a college 9 2/9 5/9 1/9 0/9 1/9 
degree in music after graduating from 
high school. 
| am interested in participating in a 9 7/9 2/9 0/9 0/9 0/9 
college band or orchestra after 
graduating from high school. 
| am interested in continuing with my 9 6/9 2/9 0/9 0/9 1/9 
instrument on an informal basis after 
graduating from high school. 
| am more motivated to do well on 9 8/9 1/9 0/9 0/9 0/9 
my instrument. 
24. Students’ report of work in school 

Strongly Strongly Don’t 

N agree Agree Disagree disagree know 
| work hard in school. 9 8/9 1/9 0/9 0/9 0/9 
Participating in the Online School 9 2/9 5/9 2/9 0/9 0/9 


Partnerships made me more 
motivated to do well in school. 
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Appendix 


Open-ended comments 


A1. Student survey open-ended comments: “What did you like most about the 
program?” 


All Schools (N=9) 


Accessibility/convenience (4) 
| like the accessibility of online lessons. 
It’s convenient. 
It’s a way for me to get lessons from a bassoon professional when there aren’t any in my area. 


| wouldn't be able to do lessons without the program (I live about an hour away from the closest private 
lessons teachers for strings). 


Skill development opportunity (4) 
The individual session allowed me to assess and improve my skills. 
Without them | would not be able to play at the level I’m at now. 
That | should get individual help on things | personally was having trouble with. 


The lessons associated with this program have helped me increase my musical range greatly and have 
made me more comfortable and brave with playing my instrument. 


Teaching Artists (2) 
Professional teachers. 
My teacher was really nice. 
Other (1) 
You could go at your own pace and not be ahead or behind others. 


Note. Responses may have been broken up to fit in respective categories. 
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A2. Student survey open-ended comments: “What would you improve about 
this class?” 


All Schools (N=9) 


Nothing (6) 
| cannot think of anything | would improve. 
There’s really not much to improve. | like it a lot. 
Nothing comes to mind. 
None 
Nothing. 
Nothing. | enjoy it a lot. 
Technology (2) 
Skype was not the best method of communication; it froze multiple times. 


Personally | would improve the technology, but that’s just because of the computer our school uses. | 
have no issues with the current program. 


Other (1) 


Price. 
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Logic model 


MacPhail Online School Partnerships Logic Model 


Overview: The MacPhail Online School Partnerships was established to partner with schools in greater Minnesota to support their music program’s 
goals by providing MacPhail faculty to teach students remotely. By teaching classes remotely, MacPhail aims to increase students’ skills, knowledge, 


and interest in music, and provide the resources for music educators to grow professionally. 


INPUTS > ACTIVITIES > OUTCOMES 


Teaching artists 


Videoconferencing 
technology 


MacPhail funding 


Other funding 


Facilities and 
physical space 


Students 


Music educators 


Teaching artists work with music educators 


to determine curriculum for students 


SHORT TERM 


LONG TERM 


Tailored lessons are offered 
to students in addition to their regular 
music classes, which include: 
- Group lessons 
- Sectionals 
- Individual lessons 


Student outcomes 


- Students show improved musical skill and 
knowledge'34 


- Students show increased interest, appreciation, 
and motivation in music? 


- Students show improved attitudes toward school? 
- Students show growth in non-academic skills? 


Students continue their 
music education 
throughout high school 
and in college® 


Student musical performances 


Music educator outcomes 
- Music educators grow as teachers* 


- Music educators have increased resources and 
capacity to grow their music programs* 


Rural students are more 
likely to have the 
opportunity to perform 
and compete at a high 
level? 


Music educators observe Teaching Artists 
and learn of new teaching approaches, 
methods, and resources 


MacPhail outcomes 


- MacPhail is able to reach out to students who would 
otherwise have no access to the organization® 


- MacPhail creates partnerships with schools 
throughout the state*> 


Partner schools 
throughout the state are 
better able to offer high- 
quality music education 
to their students® 


Evaluation and documentation 
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1 Measured by music skills assessment 
2 Measured by progress reports 
3 Measured by student survey 


4Measured by music educator survey 
5 Measured by Teaching Artist survey 
8 Measured by program records 
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Survey instruments 


MacPhail 


CENTER FOR MUSIC 
Student(s): 

School: 

Director: 

Instrument{s): 


Please place one of these numbers in each box below; then total. 5 = superior; 4 = excellent; 3= good; 2=fair; 1=poor 


Performance Factors 


MacPhail Teaching Artist: 


Comments 


Tone Quality 
Clarity, consistency, control, 
focus, warmth 


Intonation 
Accuracy, consistency of pitch 


Rhythm 

Accuracy of note and rest 
values, correctness of meters, 
duration, pulse, steadiness 


Balance Blend 
Accompaniment, awareness 
of ensemble, likeness of 
qualities, matching of 
registers (if applicable) 


Technique (facility/accuracy) 
Attacks, releases, control of 
ranges, mechanical skill 


Interpretation/Musicianship 
Dynamics, emotional 
involvement, artistry, 
ohrasing, style, tempo 


Diction — Vocal 
Bowing — Strings 
Articulation — Winds 
Execution - Percussion 


Other Performance Factors 
General conduct, 
mannerisms, poise, posture, 
etc. 


TOTAL POINTS 


Signature of Teaching Artist: 


Date: __ 
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MacPhail Online School Partnership Student Survey 
Spring 2016 


School: Grade: 


Choir/Band Teacher: Today's date: 


MacPhail Teaching Artist: Type of session (individual, sectional, or 
class): 


This survey will ask you questions about you and the music sessions you participated in this semester. We 
are interested in your honest thoughts and ideas, so there are no right or wrong answers. 


Your answers are important. Please indicate how much you agree with each of these statements and 
fill out all parts of the survey. 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 

1. | learned new musical skills that helped me in 

band/choir. Lh Le Ls LI Ls 
2. | like coming to the online sessions. LI L | L|s Ls L|s 

| liked the Teaching Artist who led the class. LI L] LJ Ls Ls 
4. Coming to the online sessions increased my 

appreciation of music. LI L? Ls Li LI 
5. The Teaching Artist used class time well. im [Jz Ls [| [Js 
6. The Teaching Artist set clear expectations for ' 2 : 

practicing and learning at home. U O LF Li U 
7. |am interested in continuing with the 

MacPhail Online Partnership program. LI L? LI Lis Ls 
8. | aminterested in continuing to develop my 

musical skills. LI Li Lis LI Ls 
9. |am interested in pursuing a college degree in ? 5 ‘ i 

music after graduating from high school. U a U Li U 
10. |am more motivated to do well on my 

instrument. LU Li Ls Li Us 
11. The technology used for these sessions made 

it easy to learn. LI Li Ls Lis Ls 
12. | would recommend these sessions to other 

students. Lh L? Ls LI Cs 
13. | work hard in school. LI L]2 L|s Ll Ls 
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15. What did you like most about this class? 


16. What would you improve about this class? 


17. Did you participate in a competition this year? 
LI' No 
Ll? Yes 


If yes, check all that apply: 
L13 — Solo/Ensemble 

L]4 Honor Band/Choir 
LIs Al-State 


THANK YOU FOR COMPLETING THE SURVEY! 
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